Input processing for computing devices with on-screen and off-screen inputs

ABSTRACT

Embodiments include apparatuses, methods, and systems for computing. An apparatus may include a display screen of a computing device, and an on-screen input interface controller co-located with the display screen. The apparatus may also include a receiver co-located with the display screen to wirelessly receive an off-screen input from an off-screen input support device of the computing device. The off-screen input support device may be separately located from the display screen. The on-screen input interface controller may process an on-screen input provided via an interaction between an on-screen input device and the display screen. In addition, the on-screen input interface controller may further process the off-screen input received by the receiver. Other embodiments may also be described and claimed.

FIELD

Embodiments of the present invention relate generally to the technicalfields of computing, and more particularly to input processing forcomputing devices with on-screen and off-screen inputs.

BACKGROUND

The background description provided herein is for the purpose ofgenerally presenting the context of the disclosure. Unless otherwiseindicated herein, the materials described in this section are not priorart to the claims in this application and are not admitted to be priorart by inclusion in this section.

A computing device, e.g., a desktop or laptop computer, a smartphone, atablet, a detachable 2-in-1 device, or an embedded device, may be ageneral-purpose device that can accept software for many purposes andapplications, or a dedicated device designed for a specific application.A computing device may include various components, e.g., a logic unit, acontrol unit, memory, and input and output devices (collectively termedI/O). Multiple I/O devices, such as a display screen, an on-screen inputdevice, e.g., a stylus, or an off-screen input device, e.g., a keyboard,may be included or coupled to a computing device. Sometimes, multipleI/O devices may be coupled to a computing device by wires, which may beexpensive to make, and not convenient to use.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments will be readily understood by the following detaileddescription in conjunction with the accompanying drawings. To facilitatethis description, like reference numerals designate like structuralelements. Embodiments are illustrated by way of example and not by wayof limitation in the figures of the accompanying drawings.

FIG. 1 illustrates an example apparatus for computing including acomputing device having an on-screen input interface controller toreceive an on-screen input from an on-screen input device, and also anoff-screen input from an off-screen input support device, in accordancewith various embodiments.

FIG. 2 illustrates another example apparatus for computing including acomputing device having an on-screen input interface controller toreceive an on-screen input from an on-screen input device, and also anoff-screen input from an off-screen input support device, in accordancewith various embodiments.

FIG. 3 illustrates another example apparatus for computing including acomputing device having an on-screen input interface controller toreceive an on-screen input from an on-screen input device, and also anoff-screen input from an off-screen input support device, in accordancewith various embodiments.

FIG. 4 illustrates another example apparatus for computing including acomputing device having an on-screen input interface controller toreceive an on-screen input from an on-screen input device, and also anoff-screen input from an off-screen input support device, in accordancewith various embodiments.

FIG. 5 illustrates another example apparatus for computing including acomputing device having an on-screen input interface controller toreceive an on-screen input from an on-screen input device, and also anoff-screen input from an off-screen input support device, in accordancewith various embodiments.

FIG. 6 illustrates an example process for an on-screen input interfacecontroller to receive an on-screen input from an on-screen input device,and also an off-screen input from an off-screen input support device, inaccordance with various embodiments.

FIG. 7 illustrates an example device suitable for use to practicevarious aspects of the present disclosure, in accordance with variousembodiments.

FIG. 8 illustrates a storage medium having instructions for practicingmethods described with references to FIGS. 1-6, in accordance withvarious embodiments.

DETAILED DESCRIPTION

Sometimes, multiple input and output (I/O) devices may be included orcoupled to a computing device by wires, which may be expensive to make,and not convenient to use. For example, a detachable 2 in 1 device mayconnect a tablet and a keyboard using pogo-pins or wire connectors. Suchconnections may have negative impacts on user experiences of the 2-in-1device, e.g., not convenient to use, not aesthetical appealing. Therehas been interest in reducing the number of pogo-pins or eliminating theuse of pogo-pins. For example, Apple iPad Pro's Smart Keyboard may havereduced the number of pogo-pins from five to six as normally used in adetachable 2-in-1 device to about three. Similarly, Google Pixel C mayprovide wireless charging and Bluetooth capability. However, the currentsolutions may be expensive, which may have limited their applications. Abetter design would benefit computing devices such as detachable 2 in 1devices, which may be an important and growing market segment in thepersonal computing (PC) industry.

Embodiments herein may present a computing device including a displayscreen. An on-screen input interface controller may be co-located withthe display screen to receive and process an on-screen input from anon-screen input device, and also wirelessly receive an off-screen inputfrom an off-screen input support device. By using one on-screen inputinterface controller to handle both on-screen input and off-screeninput, the cost for the computing device may be reduced in addition tocut the number of wired connections. The computing device may be adetachable 2 in 1 device, other general computing device, or a dedicateddevice.

In embodiments, an apparatus for computing may include a display screenof a computing device, and an on-screen input interface controllerco-located with the display screen. The apparatus may also include areceiver co-located with the display screen to wirelessly receive anoff-screen input from an off-screen input support device of thecomputing device. The off-screen input support device may be separatelylocated from the display screen. The on-screen input interfacecontroller may process an on-screen input provided via an interactionbetween an on-screen input device and the display screen. In addition,the on-screen input interface controller may further process theoff-screen input received by the receiver.

In embodiments, an apparatus for computing may include an off-screeninput device and a transmitter coupled to the off-screen input device.The off-screen input device may generate an off-screen input for acomputing device, where the off-screen input device may be separatelylocated from a display screen of the computing device. The transmittermay wirelessly transmit the off-screen input to a receiver co-locatedwith the display screen to receive the off-screen input, where theoff-screen input is to be processed by an on-screen input interfacecontroller co-located with the display screen of the computing device.

In embodiments, an apparatus for computing may include an on-screeninput interface controller co-located with a display screen of acomputing device, and a virtual device interface to operate on theon-screen input interface controller. The virtual device interface mayprocess an on-screen input provided via an interaction between anon-screen input device and the display screen; and further process anoff-screen input provided by an off-screen input support device of thecomputing device, separately located from the display screen.

In the description to follow, reference is made to the accompanyingdrawings which form a part hereof wherein like numerals designate likeparts throughout, and in which is shown by way of illustrationembodiments that may be practiced. It is to be understood that otherembodiments may be utilized and structural or logical changes may bemade without departing from the scope of the present disclosure.Therefore, the following detailed description is not to be taken in alimiting sense, and the scope of embodiments is defined by the appendedclaims and their equivalents.

Operations of various methods may be described as multiple discreteactions or operations in turn, in a manner that is most helpful inunderstanding the claimed subject matter. However, the order ofdescription should not be construed as to imply that these operationsare necessarily order dependent. In particular, these operations may notbe performed in the order of presentation. Operations described may beperformed in a different order than the described embodiments. Variousadditional operations may be performed and/or described operations maybe omitted, split or combined in additional embodiments.

For the purposes of the present disclosure, the phrase “A or B” and “Aand/or B” means (A), (B), or (A and B). For the purposes of the presentdisclosure, the phrase “A, B, and/or C” means (A), (B), (C), (A and B),(A and C), (B and C), or (A, B and C).

The description may use the phrases “in an embodiment,” or “inembodiments,” which may each refer to one or more of the same ordifferent embodiments. Furthermore, the terms “comprising,” “including,”“having,” and the like, as used with respect to embodiments of thepresent disclosure, are synonymous.

As used hereinafter, including the claims, the term “unit,” “engine,”“module,” or “routine” may refer to, be part of, or include anApplication Specific Integrated Circuit (ASIC), an electronic circuit, aprocessor (shared, dedicated, or group) and/or memory (shared,dedicated, or group) that execute one or more software or firmwareprograms, a combinational logic circuit, and/or other suitablecomponents that provide the described functionality.

Where the disclosure recites “a” or “a first” element or the equivalentthereof, such disclosure includes one or more such elements, neitherrequiring nor excluding two or more such elements. Further, ordinalindicators (e.g., first, second or third) for identified elements areused to distinguish between the elements, and do not indicate or imply arequired or limited number of such elements, nor do they indicate aparticular position or order of such elements unless otherwisespecifically stated.

The terms “coupled with” and “coupled to” and the like may be usedherein. “Coupled” may mean one or more of the following. “Coupled” maymean that two or more elements are in direct physical or electricalcontact. However, “coupled” may also mean that two or more elementsindirectly contact each other, but yet still cooperate or interact witheach other, and may mean that one or more other elements are coupled orconnected between the elements that are said to be coupled with eachother. By way of example and not limitation, “coupled” may mean two ormore elements or devices are coupled by electrical connections on aprinted circuit board such as a motherboard, for example. By way ofexample and not limitation, “coupled” may mean two or moreelements/devices cooperate and/or interact through one or more networklinkages such as wired and/or wireless networks. By way of example andnot limitation, a computing apparatus may include two or more computingdevices “coupled” on a motherboard or by one or more network linkages.

As used herein, the term “circuitry” may refer to, be part of, orinclude an Application Specific Integrated Circuit (ASIC), an electroniccircuit, a processor (shared, dedicated, or group), and/or memory(shared, dedicated, or group) that execute one or more software orfirmware programs, a combinational logic circuit, and/or other suitablehardware components that provide the described functionality. As usedherein, “computer-implemented method” may refer to any method executedby one or more processors, a computer system having one or moreprocessors, a mobile device such as a smartphone (which may include oneor more processors), a tablet, a laptop computer, a set-top box, agaming console, and so forth.

As used herein, the term “interface” or “interface circuitry” may referto, is part of, or includes circuitry providing for the exchange ofinformation between two or more components or devices. The term“interface circuitry” may refer to one or more hardware interfaces (forexample, buses, input/output (I/O) interfaces, peripheral componentinterfaces, network interface cards, and/or the like).

Some embodiments may be used in conjunction with various devices andsystems, for example, a communication system, a communication device, awireless communication system, a wireless communication device, a wiredcommunication device, a wired communication system, a personal computer(PC), a desktop computer, a mobile computer, a laptop computer, anotebook computer, an Ultrabook™ computer, a tablet computer, a servercomputer, a handheld computer, a handheld device, a Personal DigitalAssistant (PDA) device, a handheld PDA device, an on-board device, anoff-board device, a hybrid device, a vehicular device, a non-vehiculardevice, a mobile or portable device, a consumer device, a non-mobile ornon-portable device, a wireless communication station, a wirelesscommunication device, a wired or wireless modem, a video device, anaudio device, an audio-video (AN) device, a wired or wireless network, awireless area network, a wireless video area network (WVAN), a localarea network (LAN), a wireless LAN (WLAN), a personal area network(PAN), a wireless PAN (WPAN), a wireless wide area network (WWAN), andthe like.

Some embodiments may be used in conjunction with devices and/or networksoperating in accordance with existing web real-time communicationstandards, IEEE 802.11 standards, wireless-gigabit-alliance (WGA)specifications, wireless fidelity (WiFi) alliance (WFA) peer-to-peer(P2P) specifications, 3rd generation partnership project (3GPP), 3GPPlong term evolution (LTE), any current and/or future versions and/orderivatives thereof, and the like.

Some embodiments may be used in conjunction with one way and/or two-wayradio communication systems, cellular radio-telephone communicationsystems, a mobile phone, a cellular telephone, a wireless telephone, apersonal communication systems (PCS) device, a PDA device whichincorporates a wireless communication device, a mobile or portableglobal positioning system (GPS) device, a device which incorporates aGPS receiver or transceiver or chip, a device which incorporates an RFIDelement or chip, a multiple input multiple output (MIMO) transceiver ordevice, a single input multiple output (SIMO) transceiver or device, amultiple input single output (MISO) transceiver or device, a devicehaving one or more internal antennas and/or external antennas, digitalvideo broadcast (DVB) devices or systems, multi-standard radio devicesor systems, a wired or wireless handheld device, e.g., a smartphone, awireless application protocol (WAP) device, or the like.

Some embodiments may be used in conjunction with one or more types ofwireless communication signals and/or systems, for example, radiofrequency (RF), infrared (IR), frequency-division multiplexing (FDM),orthogonal FDM (OFDM), time-division multiplexing (TDM), time-divisionmultiple access (TDMA), extended TDMA (E-TDMA), general packet radioservice (GPRS), extended GPRS, code-division multiple access (CDMA),wideband CDMA (WCDMA), CDMA 2000, single-carrier CDMA, multi-carrierCDMA, multi-carrier modulation, discrete multi-tone (DMT), Bluetooth®,global positioning system (GPS), Wi-Fi, Wi-Max, ZigBee™, ultra-wideband(UWB), global system for mobile communication (GSM), 2G, 2.5G, 3G, 3.5G,4G, fifth generation (5G) mobile networks, 3GPP, long term evolution(LTE), LTE advanced, enhanced data rates for GSM evolution (EDGE), orthe like. Other embodiments may be used in various other wired and/orwireless devices, systems and/or networks.

The term “wireless device,” as used herein, includes, for example, adevice capable of wireless communication, a communication device capableof wireless communication, a communication station capable of wirelesscommunication, a portable or non-portable device capable of wirelesscommunication, or the like. In some embodiments, a wireless device maybe or may include a peripheral that is integrated with a computer, or aperipheral that is attached to a computer. In some embodiments, the term“wireless device” may optionally include a wireless service.

The term “communication device”, as used herein, includes, for example,a device capable of wireless and/or wired communication, a communicationdevice capable of wireless and/or wired communication, a communicationstation capable of wireless and/or wired communication, a portable ornon-portable device capable of wireless and/or wired communication, orthe like. In some embodiments, a communication device may be or mayinclude a peripheral that is integrated with a computer, or a peripheralthat is attached to a computer.

FIG. 1 illustrates an example apparatus 100 for computing including acomputing device 110 having an on-screen input interface controller 111to receive an on-screen input from an on-screen input device 115, andalso an off-screen input from an off-screen input support device 120, inaccordance with various embodiments. For clarity, features of theapparatus 100, the computing device 110, the on-screen input interfacecontroller 111, the on-screen input device 115, and the off-screen inputsupport device 120 may be described below as an example forunderstanding an example apparatus for computing including a computingdevice having an on-screen input interface controller to receive anon-screen input from an on-screen input device, and also an off-screeninput from an off-screen input support device. It is to be understoodthat there may be more or fewer components included in the apparatus100, the computing device 110, the on-screen input interface controller111, the on-screen input device 115, and the off-screen input supportdevice 120. Further, it is to be understood that one or more of thedevices and components within the apparatus 100, the computing device110, the on-screen input interface controller 111, the on-screen inputdevice 115, and the off-screen input support device 120 may includeadditional and/or varying features from the description below, and mayinclude any devices and components that one having ordinary skill in theart would consider and/or refer to as the devices and components of anapparatus for computing including a computing device having an on-screeninput interface controller to receive an on-screen input from anon-screen input device, and also an off-screen input from an off-screeninput support device.

In embodiments, the apparatus 100 may include the computing device 110and the off-screen input support device 120. The computing device 110may include the on-screen input interface controller 111, a displayscreen 113, the on-screen input device 115, a receiver 117, and a mainprocessor 112. The main processor 112 may perform computations for theapparatus 100 based on various inputs. The on-screen input interfacecontroller 111 and the receiver 117 may be co-located with the displayscreen 113. The off-screen input support device 120 may be separatelylocated from the display screen 113. The off-screen input support device120 may include an off-screen input device 121 and a transmitter 123. Insome embodiments, a second off-screen input device, not shown, may beincluded in the off-screen input support device 120.

In embodiments, the transmitter 123 and the receiver 117 may communicatein accordance with a wireless protocol 131. The wireless protocol 131may be selected from one of a near field communication (NFC) protocol, awireless personal area network (WPAN) protocol, a mobile body areanetworks (MBAN) protocol, an infrared protocol, a Bluetooth® protocol, aZigBee protocol, a Z-Wave protocol, a dedicated short rangecommunications (DSRC) protocol, or a wireless universal serial bus (USB)protocol.

In embodiments, an interaction between the on-screen input device 115and the display screen 113 may generate an on-screen input. Theon-screen input device 115 may be a selected one of a user finger, astylus, or a virtual keyboard. The off-screen input device 121 may be akeyboard, a trackpad, a mouse, a track ball, a joy stick, or anotherdisplay screen removably coupled with the display screen. The off-screeninput device 121 may generate an off-screen input. The transmitter 123may be coupled to the off-screen input device 121 to wirelessly transmitthe off-screen input to the receiver 117, and the receiver 117 maywirelessly receive the off-screen input from the transmitter 123. Sincethe transmitter 123 and the receiver 117 may communicate through variouswireless communication protocols, there may not be any wired connectionsbetween the computing device 110 and the off-screen input support device120. Additionally and alternatively, there may be some wires between thecomputing device 110 and the off-screen input support device 120 forfunctions such as power.

In embodiments, the on-screen input interface controller 111 may processan on-screen input provided via an interaction between the on-screeninput device 115 and the display screen 113. In addition, the on-screeninput interface controller 111 may also process the off-screen inputreceived by the receiver 117. In embodiments, the on-screen inputinterface controller 111 may receive or process in a first time slot ora first frequency the on-screen input from the on-screen input device115, and receive or process in a second time slot or a second frequencythe off-screen input from the off-screen input support device 120 andreceived by the receiver 117. By using the on-screen input interfacecontroller 111 to process both an on-screen input and an off-screeninput, the apparatus 100 may be made with less cost, and improved userexperiences.

In embodiments, the apparatus 100 may be a detachable 2 in 1 deviceincluding at least the computing device 110, the display screen 113, theon-screen input interface controller 111, and the receiver 117. Thecomputing device 110 may be a computing tablet. In some otherembodiments, the apparatus 100 may be an embedded computing device withthe display screen 113, the on-screen input device 115, and theon-screen input interface controller 111 that may further receive anoff-screen input from an additional off-screen input support device. Thedisplay screen 113 may be a selected one of a resistive touchscreen, acapacitive touchscreen, an infrared touchscreen, an optical touchscreen,a light-emitting diode (LED) display, a liquid crystal display (LCD), athin film transistor liquid crystal display (TFT-LCD), a digital lightprocessing (DLP) display, a plasma display, an electroluminescent panel,an organic light-emitting diode (OLED) display, or an electronic paper.

FIG. 2 illustrates another example apparatus 200 for computing includinga computing device 210 having an on-screen input interface controller toreceive an on-screen input from an on-screen input device 215, and alsoan off-screen input from an off-screen input support device 220, inaccordance with various embodiments. In embodiments, the apparatus 200,the computing device 210, the off-screen input support device 220, andthe on-screen input device 215, may be examples of the apparatus 100,the computing device 110, the off-screen input support device 120, andthe on-screen input device 115, respectively.

In embodiments, the apparatus 200 may include the computing device 210and the off-screen input support device 220. The apparatus 200 may be adetachable 2 in 1 device, and the computing device 210 may be acomputing tablet. The computing device 210 may include a display screen213, a receiver 217 co-located with the display screen 213, and theon-screen input device 215. The receiver 217 may be a sensing element.The on-screen input device 215 may be a stylus. An on-screen input maybe generated by pointing the stylus to be in contact with a location ofthe display screen 213. There may be a main processor and an on-screeninput interface controller, not shown, co-located with the displayscreen 213 of the computing device 210.

The off-screen input support device 220 may be a keyboard baseseparately located from the display screen 213. The off-screen inputsupport device 220 may include an off-screen input device 221 and atransmitter 223. The off-screen input support device 220 may include asecond off-screen input device 225. The transmitter 223 may also be asensing element capacitively coupled to the receiver 217. The off-screeninput device 221 may be a keyboard, and the second off-screen inputdevice 225 may be a trackpad. The off-screen input device 221 maygenerate an off-screen input by any key of the keyboard being pressed.The second off-screen input device 225 may generate an off-screen inputby moving a position in the trackpad.

In embodiments, an interaction between the on-screen input device 215and the display screen 213 may generate an on-screen input. Theoff-screen input device 221 may generate an off-screen input, while thesecond off-screen input device 225 may generate another off-screeninput. The transmitter 223 may transmit the off-screen input generatedby the off-screen input device 221 or by the second off-screen inputdevice 225 to the receiver 217 through capacitive coupling between thetransmitter 223 and the receiver 217. The receiver 217 may receive anoff-screen input from the transmitter 223 through capacitive couplingbetween the transmitter 223 and the receiver 217. Both the on-screeninput, and the one or more off-screen inputs may be processed by anon-screen input interface controller, not shown, co-located with thedisplay screen 213 of the computing device 210.

FIG. 3 illustrates another example apparatus 300 for computing includinga computing device 310 having an on-screen input interface controller toreceive an on-screen input from an on-screen input device 315, and alsoan off-screen input from an off-screen input support device 320, inaccordance with various embodiments. In embodiments, the apparatus 300,the computing device 310, the off-screen input support device 320, andthe on-screen input device 315, may be examples of the apparatus 100,the computing device 110, the off-screen input support device 120, andthe on-screen input device 115, respectively. Similarly, the apparatus300, the computing device 310, the off-screen input support device 320,and the on-screen input device 315, may be examples of the apparatus200, the computing device 210, the off-screen input support device 220,and the on-screen input device 215, respectively. In some embodiments,the apparatus 300 may be the same as the apparatus 200 in a dockedcondition.

In embodiments, the apparatus 300 may include the computing device 310and the off-screen input support device 320. The apparatus 300 may be adetachable 2 in 1 device, and the computing device 310 may be acomputing tablet. The computing device 310 may include a display screen313, the on-screen input device 315, and a receiver 317 co-located withthe display screen 313. The on-screen input device 315 may be a stylus.The off-screen input support device 320 may be a keyboard baseseparately located from the display screen 313. The off-screen inputsupport device 320 may include an off-screen input device 321 and atransmitter 323. The off-screen input device 321 may be a keyboard.

In embodiments, the transmitter 323 may be a touch sensing elementlocated in a location of the off-screen input support device 320 tocontrol a capacitance of the touch sensing element by a tilting degreeof the off-screen input support device 320 with respect to the computingdevice 310. For example, the tilting degree of the off-screen inputsupport device 320 with respect to the computing device 310 may berepresented by a degree 331 formed by an edge 329 of the off-screeninput support device 320, and an edge 319 of the computing device 310.The change of the degree 331 may affect the distance between thetransmitter 323 and the receiver 317, and hence controlling acapacitance of the transmitter 323 and the receiver 317, which are twotouch sensing elements. For example, placements of the two sensingelements, e.g., the transmitter 323 and the receiver 317, may be chosensuch that the two sensing elements may have sufficient capacitance foroperation in normal modes, and may lose the capacitance when theoff-screen input support device 320, e.g., a keyboard base, may betilted beyond a predetermined angle for operation. Such a design mayhelp making and breaking the interfaces by usage scenarios to avoidunwanted off-screen inputs, e.g., key entries, without any additionalhardware.

FIG. 4 illustrates another example apparatus 400 for computing includinga computing device 410 having an on-screen input interface controller411 to receive an on-screen input from an on-screen input device 415,and also an off-screen input from an off-screen input support device420, in accordance with various embodiments. In embodiments, theapparatus 400, the computing device 410, the off-screen input supportdevice 420, the on-screen input interface controller 411, and theon-screen input device 415, may be examples of the apparatus 100, thecomputing device 110, the off-screen input support device 120, theon-screen input interface controller 111, and the on-screen input device115, respectively. Similarly, the apparatus 400, the computing device410, the off-screen input support device 420, and the on-screen inputdevice 415, may be examples of the apparatus 200, the computing device210, the off-screen input support device 220, and the on-screen inputdevice 215, respectively.

In embodiments, the apparatus 400 may include the computing device 410and the off-screen input support device 420, which may be coupled by awired connection 433. The wired connection 433 may be for power deliverybetween the computing device 410 and the off-screen input support device420. The computing device 410 may include a display screen 413, theon-screen input device 415, the on-screen input interface controller411, and a receiver 417 co-located with the display screen 413. Theoff-screen input support device 420 may be a keyboard base separatelylocated from the display screen 413. The off-screen input support device420 may include a transmitter 423, an off-screen input device 421,another off-screen input device 422, and a main processor 427. Theoff-screen input device 421 may be a keyboard, while another off-screeninput device 422 may be a track pad. The transmitter 423 and thereceiver 417 may communicate in accordance with a wireless protocol 431.The main processor 427 may be coupled to the off-screen input device421, and another off-screen input device 422, and perform computationsfor the apparatus 400 based on various inputs.

In addition, the off-screen input support device 420 may further includean off-screen input interface controller 425 coupled to the transmitter423, the off-screen input device 421, and another off-screen inputdevice 422. The off-screen input interface controller 425 may generatethe off-screen input compatible to the on-screen input, which is to betransmitted by the transmitter 423 to the receiver 417. Furthermore, theon-screen input interface controller 411 may process an on-screen inputprovided via an interaction between the on-screen input device 415 andthe display screen 413. In addition, the on-screen input interfacecontroller 411 may also process the off-screen input generated by theoff-screen input interface controller 425 and received by the receiver417.

FIG. 5 illustrates another example apparatus 500 for computing includinga computing device 510 having an on-screen input interface controller511 to receive an on-screen input from an on-screen input device 515,and also an off-screen input from an off-screen input support device520, in accordance with various embodiments. In embodiments, theapparatus 500, the computing device 510, the off-screen input supportdevice 520, the on-screen input interface controller 511, and theon-screen input device 515, may be examples of the apparatus 100, thecomputing device 110, the off-screen input support device 120, theon-screen input interface controller 111, and the on-screen input device115, respectively. In addition to the hardware components, the apparatus500 may illustrate some functional components as well.

In embodiments, the apparatus 500 may include the computing device 510and the off-screen input support device 520. The computing device 510may include a display screen 513, the on-screen input device 515, theon-screen input interface controller 511, and a processor 512. Theoff-screen input support device 520 may be separately located from thedisplay screen 513, and may include an off-screen input device 521. Inaddition, the computing device 510 may include a receiver, and theoff-screen input support device 520 may include a transmitter tocommunicate with the receiver in accordance with a wireless protocol,not shown.

In addition to the hardware components, the computing device 510 mayinclude various software components, e.g., a firmware 530, an operatingsystem 540, and applications 551 running on the on-screen inputinterface controller 511 and the processor 512. The firmware 530 mayinclude a virtual device interface 535 to operate on the on-screen inputinterface controller 511. The operating system 540 may include anon-screen input device driver 543 to operate on the processor 512. Thevirtual device interface 535 may process an on-screen input provided viaan interaction between the on-screen input device 515 and the displayscreen 513. In addition, the virtual device interface 535 may process anoff-screen input provided by the off-screen input support device 520.The virtual device interface 535 may process the on-screen input in afirst time slot or a first frequency, and process the off-screen inputin a second time slot or a second frequency.

FIG. 6 illustrates an example process 600 for an on-screen inputinterface controller to receive an on-screen input from an on-screeninput device, and also an off-screen input from an off-screen inputsupport device, in accordance with various embodiments. In embodiments,the process 600 may be a process performed by the on-screen inputinterface controller 111 in FIG. 1, the on-screen input interfacecontroller 411 in FIG. 4, or the on-screen input interface controller511 in FIG. 5.

The process 600 may start at an interaction 601. During the interaction601, an on-screen input may be received, where the on-screen input maybe provided via an interaction between an on-screen input device and adisplay screen of a computing device. For example, during theinteraction 601, an on-screen input may be received, where the on-screeninput may be provided via an interaction between the on-screen inputdevice 115 and the display screen 113 of the computing device 110.

During an interaction 603, the on-screen input may be processed by anon-screen input interface controller co-located with the display screen.For example, during the interaction 603, the on-screen input may beprocessed by the on-screen input interface controller 111 co-locatedwith the display screen 113.

During an interaction 605, an off-screen input may be received by areceiver co-located with the display screen in accordance with awireless protocol, where the off-screen input may be generated by anoff-screen input device of the computing device separately located fromthe display screen. For example, during the interaction 605, anoff-screen input may be received by the receiver 117 co-located with thedisplay screen 113 in accordance with the wireless protocol 131, wherethe off-screen input may be generated by the off-screen input device 121of the computing device separately located from the display screen 113.

During an interaction 607, the off-screen input may be processed by theon-screen input interface controller coupled to the receiver. Forexample, during the interaction 607, the off-screen input may beprocessed by the on-screen input interface controller 111 coupled to thereceiver 117.

FIG. 7 illustrates an example device suitable for use to practicevarious aspects of the present disclosure, in accordance with variousembodiments. The device 700 may be used to implement functions of theapparatus 100, the apparatus 200, the apparatus 300, the apparatus 400,or the apparatus 500. As shown, the device 700 may include one or moreprocessors 702, each having one or more processor cores, or andoptionally, a hardware accelerator 703 (which may be an ASIC or a FPGA).In alternate embodiments, the hardware accelerator 703 may be part ofprocessor 702, or integrated together on a SOC. Additionally, the device700 may include a memory 704, which may be any one of a number of knownpersistent storage medium, and a data storage circuitry 708 includingmodules 709. In addition, the 700 may include an I/O interface 718,coupled to one or more sensors 714. Furthermore, the device 700 mayinclude communication circuitry 705 including a transceiver (Tx) 711,and network interface controller (NIC) 712. The elements may be coupledto each other via system bus 706, which may represent one or more buses.In the case of multiple buses, they may be bridged by one or more busbridges (not shown).

In addition, the device 700 may include a display screen 713, anon-screen input device 715, an on-screen input interface controller 711,and an off-screen input device 721. Furthermore, the I/O interface 718may include a transmitter 723 and a receiver 717. In embodiments, thedisplay screen 713, the on-screen input device 715, the on-screen inputinterface controller 711, the off-screen input device 721, thetransmitter 723, and the receiver 717 may be similar to thecorresponding components, e.g., the display screen 113, the on-screeninput device 115, the on-screen input interface controller 111, theoff-screen input device 121, the transmitter 123, and the receiver 117,as shown in FIG. 1.

In embodiments, the processor(s) 702 (also referred to as “processorcircuitry 702”) may be one or more processing elements configured toperform basic arithmetical, logical, and input/output operations bycarrying out instructions. Processor circuitry 702 may be implemented asa standalone system/device/package or as part of an existingsystem/device/package. The processor circuitry 702 may be one or moremicroprocessors, one or more single-core processors, one or moremulti-core processors, one or more multithreaded processors, one or moreGPUs, one or more ultra-low voltage processors, one or more embeddedprocessors, one or more DSPs, one or more FPDs (hardware accelerators)such as FPGAs, structured ASICs, programmable SoCs (PSoCs), etc., and/orother processor or processing/controlling circuit. The processorcircuitry 702 may be a part of a SoC in which the processor circuitry702 and other components discussed herein are formed into a single IC ora single package. As examples, the processor circuitry 702 may includeone or more Intel Pentium®, Core®, Xeon®, Atom®, or Core M®processor(s); Advanced Micro Devices (AMD) Accelerated Processing Units(APUs), Epyc®, or Ryzen® processors; Apple Inc. A series, S series, Wseries, etc. processor(s); Qualcomm snapdragon® processor(s); SamsungExynos® processor(s); and/or the like.

In embodiments, the processor circuitry 702 may include a sensor hub,which may act as a coprocessor by processing data obtained from the oneor more sensors 714. The sensor hub may include circuitry configured tointegrate data obtained from each of the one or more sensors 714 byperforming arithmetical, logical, and input/output operations. Inembodiments, the sensor hub may capable of timestamping obtained sensordata, providing sensor data to the processor circuitry 702 in responseto a query for such data, buffering sensor data, continuously streamingsensor data to the processor circuitry 702 including independent streamsfor each sensor of the one or more sensors 714, reporting sensor databased upon predefined thresholds or conditions/triggers, and/or otherlike data processing functions.

In embodiments, the memory 704 (also referred to as “memory circuitry704” or the like) may be circuitry configured to store data or logic foroperating the computer device 700. The memory circuitry 704 may includenumber of memory devices may be used to provide for a given amount ofsystem memory. As examples, the memory circuitry 704 can be any suitabletype, number and/or combination of volatile memory devices (e.g., randomaccess memory (RAM), dynamic RAM (DRAM), static RAM (SAM), etc.) and/ornon-volatile memory devices (e.g., read-only memory (ROM), erasableprogrammable read-only memory (EPROM), electrically erasableprogrammable read-only memory (EEPROM), flash memory, antifuses, etc.)that may be configured in any suitable implementation as are known. Invarious implementations, individual memory devices may be formed of anynumber of different package types, such as single die package (SDP),dual die package (DDP) or quad die package (Q17P), dual inline memorymodules (DIMMs) such as microDIMMs or MiniDIMMs, and/or any other likememory devices. To provide for persistent storage of information such asdata, applications, operating systems and so forth, the memory circuitry704 may include one or more mass-storage devices, such as a solid statedisk drive (SSDD); flash memory cards, such as SD cards, microSD cards,xD picture cards, and the like, and USB flash drives; on-die memory orregisters associated with the processor circuitry 702 (for example, inlow power implementations); a micro hard disk drive (HDD); threedimensional cross-point (3D XPOINT) memories from Intel® and Micron®,etc.

Where FPDs are used, the processor circuitry 702 and memory circuitry704 (and/or data storage circuitry 708) may comprise logic blocks orlogic fabric, memory cells, input/output (I/O) blocks, and otherinterconnected resources that may be programmed to perform variousfunctions of the example embodiments discussed herein. The memory cellsmay be used to store data in lookup-tables (LUTs) that are used by theprocessor circuitry 702 to implement various logic functions. The memorycells may include any combination of various levels of memory/storageincluding, but not limited to, EPROM, EEPROM, flash memory, SRAM,anti-fuses, etc.

In embodiments, the data storage circuitry 708 (also referred to as“storage circuitry 708” or the like), with shared or respectivecontrollers, may provide for persistent storage of information such asmodules 709, operating systems, etc. The data storage circuitry 708 maybe implemented as solid state drives (SSDs); solid state disk drive(SSDD); serial AT attachment (SATA) storage devices (e.g., SATA SSDs);flash drives; flash memory cards, such as SD cards, microSD cards, xDpicture cards, and the like, and USB flash drives; three-dimensionalcross-point (3D Xpoint) memory devices; on-die memory or registersassociated with the processor circuitry 702; hard disk drives (HDDs);micro HDDs; resistance change memories; phase change memories;holographic memories; or chemical memories; among others. As shown, thedata storage circuitry 708 is included in the computer device 700;however, in other embodiments, the data storage circuitry 708 may beimplemented as one or more devices separated from the other elements ofcomputer device 700.

In some embodiments, the data storage circuitry 708 may include anoperating system (OS) (not shown), which may be a general purposeoperating system or an operating system specifically written for andtailored to the computer device 700. The OS may include one or moredrivers, libraries, and/or application programming interfaces (APIs),which provide program code and/or software components for modules 709and/or control system configurations to control and/or obtain/processdata from the one or more sensors 714.

The modules 709 may be software modules/components used to performvarious functions of the computer device 700 and/or to carry outfunctions of the example embodiments discussed herein. In embodimentswhere the processor circuitry 702 and memory circuitry 704 includeshardware accelerators (e.g., FPGA cells, the hardware accelerator 703)as well as processor cores, the hardware accelerators (e.g., the FPGAcells) may be pre-configured (e.g., with appropriate bit streams, logicblocks/fabric, etc.) with the logic to perform some functions of theembodiments herein (in lieu of employment of programming instructions tobe executed by the processor core(s)). For example, the modules 709 maycomprise logic for the corresponding entities discussed with regard tothe display screen 713, the on-screen input device 715, the on-screeninput interface controller 711, the off-screen input device 721, thetransmitter 723, and the receiver 717.

The components of computer device 700 may communicate with one anotherover the bus 706. The bus 706 may include any number of technologies,such as a Local Interconnect Network (LIN); industry standardarchitecture (ISA); extended ISA (EISA); PCI; PCI extended (PCIx); PCIe;an Inter-Integrated Circuit (I2C) bus; a Parallel Small Computer SystemInterface (SPI) bus; Common Application Programming Interface (CAPI);point to point interfaces; a power bus; a proprietary bus, for example,Intel® Ultra Path Interface (UPI), Intel® Accelerator Link (IAL), orsome other proprietary bus used in a SoC based interface; or any numberof other technologies. In some embodiments, the bus 706 may be acontroller area network (CAN) bus system, a Time-Trigger Protocol (TTP)system, or a FlexRay system, which may allow various devices (e.g., theone or more sensors 714, etc.) to communicate with one another usingmessages or frames.

The communications circuitry 705 may include circuitry for communicatingwith a wireless network or wired network. For example, the communicationcircuitry 705 may include transceiver (Tx) 711 and network interfacecontroller (NIC) 712. Communications circuitry 705 may include one ormore processors (e.g., baseband processors, modems, etc.) that arededicated to a particular wireless communication protocol.

NIC 712 may be included to provide a wired communication link to anetwork and/or other devices. The wired communication may provide anEthernet connection, an Ethernet-over-USB, and/or the like, or may bebased on other types of networks, such as DeviceNet, ControlNet, DataHighway+, PROFIBUS, or PROFINET, among many others. An additional NIC712 may be included to allow connect to a second network (not shown) orother devices, for example, a first NIC 712 providing communications tothe network 150 over Ethernet, and a second NIC 712 providingcommunications to other devices over another type of network, such as apersonal area network (PAN) including a personal computer (PC) device.In some embodiments, the various components of the device 700, such asthe one or more sensors 714, etc. may be connected to the processor(s)702 via the NIC 712 as discussed above rather than via the I/O circuitry718 as discussed infra.

The Tx 711 may include one or more radios to wirelessly communicate witha network and/or other devices. The Tx 711 may include hardware devicesthat enable communication with wired networks and/or other devices usingmodulated electromagnetic radiation through a solid or non-solid medium.Such hardware devices may include switches, filters, amplifiers, antennaelements, and the like to facilitate the communications over the air(OTA) by generating or otherwise producing radio waves to transmit datato one or more other devices, and converting received signals intousable information, such as digital data, which may be provided to oneor more other components of computer device 700. In some embodiments,the various components of the device 700, such as the one or moresensors 714, etc. may be connected to the device 700 via the Tx 711 asdiscussed above rather than via the I/O circuitry 718 as discussedinfra. In one example, the one or more sensors 714 may be coupled withdevice 700 via a short range communication protocol.

The Tx 711 may include one or multiple radios that are compatible withany number of 3GPP (Third Generation Partnership Project)specifications, notably Long Term Evolution (LTE), Long TermEvolution-Advanced (LTE-A), Long Term Evolution-Advanced Pro (LTE-APro), and Fifth Generation (5G) New Radio (NR). It can be noted thatradios compatible with any number of other fixed, mobile, or satellitecommunication technologies and standards may be selected. These mayinclude, for example, any Cellular Wide Area radio communicationtechnology, which may include e.g. a 5G communication systems, a GlobalSystem for Mobile Communications (GSM) radio communication technology, aGeneral Packet Radio Service (GPRS) radio communication technology, oran Enhanced Data Rates for GSM Evolution (EDGE) radio communicationtechnology. Other Third Generation Partnership Project (3GPP) radiocommunication technology that may be used includes UMTS (UniversalMobile Telecommunications System), FOMA (Freedom of Multimedia Access),3GPP LTE (Long Term Evolution), 3GPP LTE Advanced (Long Term EvolutionAdvanced), 3GPP LTE Advanced Pro (Long Term Evolution Advanced Pro)),CDMA2000 (Code division multiple access 2000), CDPD (Cellular DigitalPacket Data), Mobitex, 3G (Third Generation), CSD (Circuit SwitchedData), HSCSD (High-Speed Circuit-Switched Data), UMTS (3G) (UniversalMobile Telecommunications System (Third Generation)), W-CDMA (UMTS)(Wideband Code Division Multiple Access (Universal MobileTelecommunications System)), HSPA (High Speed Packet Access), HSDPA(High-Speed Downlink Packet Access), HSUPA (High-Speed Uplink PacketAccess), HSPA+ (High Speed Packet Access Plus), UMTS-TDD (UniversalMobile Telecommunications System-Time-Division Duplex), TD-CDMA (TimeDivision-Code Division Multiple Access), TD-SCDMA (TimeDivision-Synchronous Code Division Multiple Access), 3GPP Rel. 8(Pre-4G) (3rd Generation Partnership Project Release 8 (Pre-4thGeneration)), 3GPP Rel. 9 (3rd Generation Partnership Project Release9), 3GPP Rel. 10 (3rd Generation Partnership Project Release 10), 3GPPRel. 11 (3rd Generation Partnership Project Release 11), 3GPP Rel. 12(3rd Generation Partnership Project Release 12), 3GPP Rel. 13 (3rdGeneration Partnership Project Release 13), 3GPP Rel. 14 (3rd GenerationPartnership Project Release 14), 3GPP LTE Extra, LTE Licensed-AssistedAccess (LAA), UTRA (UMTS Terrestrial Radio Access), E-UTRA (Evolved UMTSTerrestrial Radio Access), LTE Advanced (4G) (Long Term EvolutionAdvanced (4th Generation)), cdmaOne (2G), CDMA2000 (3G) (Code divisionmultiple access 2000 (Third generation)), EV-DO (Evolution-DataOptimized or Evolution-Data Only), AMPS (1G) (Advanced Mobile PhoneSystem (1st Generation)), TACS/ETACS (Total Access CommunicationSystem/Extended Total Access Communication System), D-AMPS (2G) (DigitalAMPS (2nd Generation)), PTT (Push-to-talk), MTS (Mobile TelephoneSystem), IMTS (Improved Mobile Telephone System), AMTS (Advanced MobileTelephone System), OLT (Norwegian for Offentlig Landmobil Telefoni,Public Land Mobile Telephony), MTD (Swedish abbreviation forMobiltelefonisystem D, or Mobile telephony system D), Autotel/PALM(Public Automated Land Mobile), ARP (Finnish for Autoradiopuhelin, “carradio phone”), NMT (Nordic Mobile Telephony), Hicap (High capacityversion of NTT (Nippon Telegraph and Telephone)), CDPD (Cellular DigitalPacket Data), Mobitex, DataTAC, iDEN (Integrated Digital EnhancedNetwork), PDC (Personal Digital Cellular), CSD (Circuit Switched Data),PHS (Personal Handy-phone System), WiDEN (Wideband Integrated DigitalEnhanced Network), iBurst, Unlicensed Mobile Access (UMA, also referredto as also referred to as 3GPP Generic Access Network, or GANstandard)), Wireless Gigabit Alliance (WiGig) standard, mmWave standardsin general (wireless systems operating at 10-90 GHz and above such asWiGig, IEEE 802.11ad, IEEE 802.11ay, and the like. In addition to thestandards listed above, any number of satellite uplink technologies maybe used for the uplink transceiver, including, for example, radioscompliant with standards issued by the ITU (InternationalTelecommunication Union), or the ETSI (European TelecommunicationsStandards Institute), among others. The examples provided herein arethus understood as being applicable to various other communicationtechnologies, both existing and not yet formulated. Implementations,components, and details of the aforementioned protocols may be thoseknown in the art and are omitted herein for the sake of brevity.

The input/output (I/O) interface 718 may include circuitry, such as anexternal expansion bus (e.g., Universal Serial Bus (USB), FireWire,Thunderbolt, PCI/PCIe/PCIx, etc.), used to connect computer device 700with external components/devices, such as one or more sensors 714, etc.I/O interface circuitry 718 may include any suitable interfacecontrollers and connectors to interconnect one or more of the processorcircuitry 702, memory circuitry 704, data storage circuitry 708,communication circuitry 705, and the other components of computer device700. The interface controllers may include, but are not limited to,memory controllers, storage controllers (e.g., redundant array ofindependent disk (RAID) controllers, baseboard management controllers(BMCs), input/output controllers, host controllers, etc. The connectorsmay include, for example, busses (e.g., bus 706), ports, slots, jumpers,interconnect modules, receptacles, modular connectors, etc. The I/Ocircuitry 718 may couple the device 700 with the one or more sensors714, etc. via a wired connection, such as using USB, FireWire,Thunderbolt, RCA, a video graphics array (VGA), a digital visualinterface (DVI) and/or mini-DVI, a high-definition multimedia interface(HDMI), an S-Video, and/or the like.

The one or more sensors 714 may be any device configured to detectevents or environmental changes, convert the detected events intoelectrical signals and/or digital data, and transmit/send thesignals/data to the computer device 700. Some of the one or more sensors714 may be sensors used for providing computer-generated sensory inputs.Some of the one or more sensors 714 may be sensors used for motionand/or object detection. Examples of such one or more sensors 714 mayinclude, inter alia, charged-coupled devices (CCD), Complementarymetal-oxide-semiconductor (CMOS) active pixel sensors (APS), lens-lessimage capture devices/cameras, thermographic (infrared) cameras, LightImaging Detection And Ranging (LIDAR) systems, and/or the like. In someimplementations, the one or more sensors 714 may include a lens-lessimage capture mechanism comprising an array of aperture elements,wherein light passing through the array of aperture elements define thepixels of an image. In embodiments, the motion detection one or moresensors 714 may be coupled with or associated with light generatingdevices, for example, one or more infrared projectors to project a gridof infrared light onto a scene, where an infrared camera may recordreflected infrared light to compute depth information.

Some of the one or more sensors 714 may be used for position and/ororientation detection, ambient/environmental condition detection, andthe like. Examples of such one or more sensors 714 may include, interalia, microelectromechanical systems (MEMS) with piezoelectric,piezoresistive and/or capacitive components, which may be used todetermine environmental conditions or location information related tothe computer device 700. In embodiments, the MEMS may include 3-axisaccelerometers, 3-axis gyroscopes, and/or magnetometers. In someembodiments, the one or more sensors 714 may also include one or moregravimeters, altimeters, barometers, proximity sensors (e.g., infraredradiation detector(s) and the like), depth sensors, ambient lightsensors, thermal sensors (thermometers), ultrasonic transceivers, and/orthe like.

Each of these elements, e.g., one or more processors 702, the hardwareaccelerator 703, the memory 704, the data storage circuitry 708including the modules 709, the input/output interface 718, the one ormore sensors 714, the communication circuitry 705 including the Tx 711,and the NIC 712, and the system bus 706, may perform its conventionalfunctions known in the art. In addition, they may be employed to storeand host execution of programming instructions implementing theoperations associated with operations to be performed by an apparatusfor computer assisted or autonomous driving, as described in connectionwith FIGS. 1-6, and/or other functions that provides the capability ofthe embodiments described in the current disclosure. The variouselements may be implemented by assembler instructions supported byprocessor(s) 702 or high-level languages, such as, for example, C, thatcan be compiled into such instructions. Operations associated with thedevice 700 not implemented in software may be implemented in hardware,e.g., via hardware accelerator 703.

The number, capability and/or capacity of these elements 702-723 mayvary, depending on the number of other devices the device 700 isconfigured to support. Otherwise, the constitutions of elements 702-723are known, and accordingly will not be further described.

As will be appreciated by one skilled in the art, the present disclosuremay be embodied as methods or computer program products. Accordingly,the present disclosure, in addition to being embodied in hardware asearlier described, may take the form of an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to as a “circuit,” “module,” or “system.”

Furthermore, the present disclosure may take the form of a computerprogram product embodied in any tangible or non-transitory medium ofexpression having computer-usable program code embodied in the medium.FIG. 8 illustrates an example computer-readable non-transitory storagemedium that may be suitable for use to store instructions that cause anapparatus, in response to execution of the instructions by theapparatus, to practice selected aspects of the present disclosure. Asshown, non-transitory computer-readable storage medium 802 may include anumber of programming instructions 804. Programming instructions 804 maybe configured to enable a device, e.g., device 800, in response toexecution of the programming instructions, to perform, e.g., variousoperations associated with an apparatus for computing including acomputing device having an on-screen input interface controller toreceive an on-screen input from an on-screen input device, and also anoff-screen input from an off-screen input support device, as shown inFIGS. 1-7.

In alternate embodiments, programming instructions 804 may be disposedon multiple computer-readable non-transitory storage media 802 instead.In alternate embodiments, programming instructions 804 may be disposedon computer-readable transitory storage media 802, such as, signals. Anycombination of one or more computer usable or computer readablemedium(s) may be utilized. The computer-usable or computer-readablemedium may be, for example but not limited to, an electronic, magnetic,optical, electromagnetic, infrared, or semiconductor system, apparatus,device, or propagation medium. More specific examples (a non-exhaustivelist) of the computer-readable medium would include the following: anelectrical connection having one or more wires, a portable computerdiskette, a hard disk, a random access memory (RAM), a read-only memory(ROM), an erasable programmable read-only memory (EPROM or Flashmemory), an optical fiber, a portable compact disc read-only memory(CD-ROM), an optical storage device, a transmission media such as thosesupporting the Internet or an intranet, or a magnetic storage device.Note that the computer-usable or computer-readable medium could even bepaper or another suitable medium upon which the program is printed, asthe program can be electronically captured, via, for instance, opticalscanning of the paper or other medium, then compiled, interpreted, orotherwise processed in a suitable manner, if necessary, and then storedin a computer memory. In the context of this document, a computer-usableor computer-readable medium may be any medium that can contain, store,communicate, propagate, or transport the program for use by or inconnection with the instruction execution system, apparatus, or device.The computer-usable medium may include a propagated data signal with thecomputer-usable program code embodied therewith, either in baseband oras part of a carrier wave. The computer usable program code may betransmitted using any appropriate medium, including but not limited towireless, wireline, optical fiber cable, RF, etc.

Computer program code for carrying out operations of the presentdisclosure may be written in any combination of one or more programminglanguages, including an object oriented programming language such asJava, Smalltalk, C++ or the like and conventional procedural programminglanguages, such as the “C” programming language or similar programminglanguages. The program code may execute entirely on the user's computer,partly on the user's computer, as a stand-alone software package, partlyon the user's computer and partly on a remote computer or entirely onthe remote computer or server. In the latter scenario, the remotecomputer may be connected to the user's computer through any type ofnetwork, including a local area network (LAN) or a wide area network(WAN), or the connection may be made to an external computer (forexample, through the Internet using an Internet Service Provider).

The present disclosure is described with reference to flowchartillustrations and/or block diagrams of methods, apparatus (systems) andcomputer program products according to embodiments of the disclosure. Itwill be understood that each block of the flowchart illustrations and/orblock diagrams, and combinations of blocks in the flowchartillustrations and/or block diagrams, can be implemented by computerprogram instructions. These computer program instructions may beprovided to a processor of a general purpose computer, special purposecomputer, or other programmable data processing apparatus to produce amachine, such that the instructions, which execute via the processor ofthe computer or other programmable data processing apparatus, createmeans for implementing the functions/acts specified in the flowchartand/or block diagram block or blocks.

These computer program instructions may also be stored in acomputer-readable medium that can direct a computer or otherprogrammable data processing apparatus to function in a particularmanner, such that the instructions stored in the computer-readablemedium produce an article of manufacture including instruction meanswhich implement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer orother programmable data processing apparatus to cause a series ofoperational steps to be performed on the computer or other programmableapparatus to produce a computer implemented process such that theinstructions which execute on the computer or other programmableapparatus provide processes for implementing the functions/actsspecified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present disclosure. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions. As used herein,“computer-implemented method” may refer to any method executed by one ormore processors, a computer system having one or more processors, amobile device such as a smartphone (which may include one or moreprocessors), a tablet, a laptop computer, a set-top box, a gamingconsole, and so forth.

Embodiments may be implemented as a computer process, a computing systemor as an article of manufacture such as a computer program product ofcomputer readable media. The computer program product may be a computerstorage medium readable by a computer system and encoding a computerprogram instructions for executing a computer process.

The corresponding structures, material, acts, and equivalents of allmeans or steps plus function elements in the claims below are intendedto include any structure, material or act for performing the function incombination with other claimed elements are specifically claimed. Thedescription of the present disclosure has been presented for purposes ofillustration and description, but is not intended to be exhaustive orlimited to the disclosure in the form disclosed. Many modifications andvariations will be apparent to those of ordinary skill without departingfrom the scope and spirit of the disclosure. The embodiment are chosenand described in order to best explain the principles of the disclosureand the practical application, and to enable others of ordinary skill inthe art to understand the disclosure for embodiments with variousmodifications as are suited to the particular use contemplated.

Thus various example embodiments of the present disclosure have beendescribed including, but are not limited to:

Example 1 may include an apparatus for computing, comprising: anon-screen input interface controller co-located with a display screen ofa computing device, to process an on-screen input provided via aninteraction between an on-screen input device and the display screen;and a receiver co-located with the display screen to wirelessly receivean off-screen input from an off-screen input support device of thecomputing device, separately located from the display screen; whereinthe on-screen input interface controller is to further process theoff-screen input received by the receiver.

Example 2 may include the apparatus of example 1 and/or some otherexamples herein, wherein the receiver is to wirelessly receive theoff-screen input in accordance with a wireless protocol selected fromone of a near field communication (NFC) protocol, a wireless personalarea network (WPAN) protocol, a mobile body area networks (MBAN)protocol, an infrared protocol, a Bluetooth® protocol, a ZigBeeprotocol, a Z-Wave protocol, a dedicated short range communications(DSRC) protocol, or a wireless universal serial bus (USB) protocol.

Example 3 may include the apparatus of example 1 and/or some otherexamples herein, wherein the on-screen input interface controller is toreceive the on-screen input in a first time slot or a first frequency,and to receive the off-screen input in a second time slot or a secondfrequency.

Example 4 may include the apparatus of example 1 and/or some otherexamples herein, wherein the display screen is a selected one of aresistive touchscreen, a capacitive touchscreen, an infraredtouchscreen, an optical touchscreen, a light-emitting diode (LED)display, a liquid crystal display (LCD), a thin film transistor liquidcrystal display (TFT-LCD), a digital light processing (DLP) display, aplasma display, an electroluminescent panel, an organic light-emittingdiode (OLED) display, or an electronic paper.

Example 5 may include the apparatus of example 1 and/or some otherexamples herein, wherein the on-screen input device is a selected one ofa user finger, a stylus, or a virtual keyboard.

Example 6 may include the apparatus of any one of examples 1-5 and/orsome other examples herein, wherein the off-screen input support deviceincludes: an off-screen input device to generate the off-screen input,and a transmitter coupled to the off-screen input device to wirelesslytransmit the off-screen input to the receiver.

Example 7 may include the apparatus of example 6 and/or some otherexamples herein, wherein the off-screen input device is a selected oneof a keyboard, a trackpad, a mouse, a track ball, a joy stick, oranother display screen removably coupled with the display screen.

Example 8 may include the apparatus of example 6 and/or some otherexamples herein, wherein the off-screen input device is a firstoff-screen input device, and the off-screen input support device furtherincludes: a second off-screen input device, wherein the secondoff-screen input device is to generate a second off-screen input to betransmitted by the transmitter to the receiver through the wirelessprotocol.

Example 9 may include the apparatus of example 6 and/or some otherexamples herein, wherein the off-screen input support device furtherincludes an off-screen input interface controller coupled to theoff-screen input device and the transmitter, and the off-screen inputinterface controller is to generate the off-screen input compatible tothe on-screen input.

Example 10 may include the apparatus of example 6 and/or some otherexamples herein, wherein the transmitter is a touch sensing elementlocated in a location of the off-screen input support device to controla capacitance of the touch sensing element by a tilting degree of theoff-screen input support device with respect to the computing device.

Example 11 may include the apparatus of example 6 and/or some otherexamples herein, wherein the transmitter is a first sensing element, thereceiver is a second sensing element capacitively coupled to the firstsensing element, the transmitter is to transmit the off-screen input tothe receiver through capacitive coupling between the first sensingelement and the second sensing element.

Example 12 may include the apparatus of example 6 and/or some otherexamples herein, wherein the computing device is a computing tablet, theoff-screen input support device is a keyboard base, the on-screen inputdevice is a stylus, the off-screen input device is a keyboard, and thetransmitter and the receiver are two touch sensing elements.

Example 13 may include the apparatus of any one of examples 1-5 and/orsome other examples herein, wherein the apparatus is the computingdevice having at least the display screen, the on-screen input interfacecontroller, and the receiver, or the computing device having at least amain processor located at the off-screen input support device.

Example 14 may include the apparatus of example 13 and/or some otherexamples herein, wherein the computing device is a computing tablet.

Example 15 may include the apparatus of any one of examples 1-5 and/orsome other examples herein, wherein the apparatus is a detachable 2 in 1device comprising at least the computing device, the display screen, theon-screen input interface controller, and the receiver.

Example 16 may include an apparatus for computing, comprising: anoff-screen input device to generate an off-screen input for a computingdevice, wherein the off-screen input device is separately located from adisplay screen of the computing device; a transmitter coupled to theoff-screen input device to wirelessly transmit the off-screen input to areceiver co-located with the display screen to receive the off-screeninput, wherein the off-screen input is to be processed by an on-screeninput interface controller co-located with the display screen of thecomputing device.

Example 17 may include the apparatus of example 16 and/or some otherexamples herein, wherein the on-screen input interface controller isfurther to process an on-screen input provided via an interactionbetween an on-screen input device and the display screen.

Example 18 may include the apparatus of example 16 and/or some otherexamples herein, wherein the off-screen input device is a firstoff-screen input device, and the apparatus further includes: a secondoff-screen input device, wherein the second off-screen input device isto generate a second off-screen input to be transmitted by thetransmitter to the receiver through the wireless protocol.

Example 19 may include the apparatus of example 16 and/or some otherexamples herein, further comprising: an off-screen input interfacecontroller coupled to the off-screen input device and the transmitter,and the off-screen input interface controller is to generate theoff-screen input compatible to the on-screen input interface controller.

Example 20 may include the apparatus of example 16 and/or some otherexamples herein, wherein the off-screen input device is a selected oneof a keyboard, a trackpad, a mouse, a track ball, a joy stick, oranother display screen removably coupled with the display screen.

Example 21 may include the apparatus of example 16 and/or some otherexamples herein, wherein the computing device is a computing tablet, theapparatus is a keyboard base, the off-screen input device is a keyboard,and the transmitter and the receiver are two touch sensing elements.

Example 22 may include the apparatus of example 16 and/or some otherexamples herein, wherein the computing device has at least a mainprocessor coupled to the off-screen input device.

Example 23 may include the apparatus of any one of examples 16-22 and/orsome other examples herein, wherein the apparatus is a detachable 2 in 1device comprising at least the computing device, the display screen, theon-screen input interface controller, the transmitter, and theoff-screen input device.

Example 24 may include an apparatus for computing, comprising: anon-screen input interface controller co-located with a display screen ofa computing device, and a virtual device interface to operate on theon-screen input interface controller to: process an on-screen inputprovided via an interaction between an on-screen input device and thedisplay screen; and process an off-screen input provided by anoff-screen input support device of the computing device, separatelylocated from the display screen.

Example 25 may include the apparatus of example 24 and/or some otherexamples herein, wherein the virtual device interface is to process theon-screen input in a first time slot or a first frequency, and toprocess the off-screen input in a second time slot or a secondfrequency.

Example 26 may include the apparatus of example 24 and/or some otherexamples herein, wherein the virtual device interface is a firmware tooperate on the on-screen input interface controller.

Example 27 may include the apparatus of example 24 and/or some otherexamples herein, wherein the on-screen input device is a selected one ofa user finger, a stylus, or a virtual keyboard.

Example 28 may include the apparatus of any one of examples 24-27 and/orsome other examples herein, wherein the off-screen input support deviceincludes: an off-screen input device to generate the off-screen input,and a transmitter coupled to the off-screen input device to wirelesslytransmit the off-screen input to a receiver co-located with the displayscreen.

Example 29 may include the apparatus of example 28 and/or some otherexamples herein, wherein the off-screen input device is a selected oneof a keyboard, a trackpad, a mouse, a track ball, a joy stick, oranother display screen removably coupled with the display screen.

Example 30 may include the apparatus of example 28 and/or some otherexamples herein, wherein the computing device is a computing tablet, theoff-screen input support device is a keyboard base, the on-screen inputdevice is a stylus, the off-screen input device is a keyboard, and thetransmitter and the receiver are two touch sensing elements.

Example 31 may include the apparatus of example 28 and/or some otherexamples herein, wherein the apparatus is the computing device having atleast the display screen, the on-screen input interface controller, andthe receiver.

Example 32 may include the apparatus of example 28 and/or some otherexamples herein, wherein the computing device is a computing tablet.

Example 33 may include the apparatus of any one of examples 24-27 and/orsome other examples herein, wherein the apparatus is a detachable 2 in 1device comprising at least the computing device, the display screen, andthe on-screen input interface controller.

Example 34 may include a method for processing inputs to a computingsystem, comprising: receiving an on-screen input provided via aninteraction between an on-screen input device and a display screen of acomputing device; processing, by an on-screen input interface controllerco-located with the display screen, the on-screen input; receiving, by areceiver co-located with the display screen, in accordance with awireless protocol, an off-screen input, wherein the off-screen input isgenerated by an off-screen input device of the computing device,separately located from the display screen; and processing, by theon-screen input interface controller coupled to the receiver, theoff-screen input.

Example 35 may include the method of example 34 and/or some otherexamples herein, wherein the wireless protocol includes a selected oneof a near field communication (NFC) protocol, a wireless personal areanetwork (WPAN) protocol, a mobile body area networks (MBAN) protocol, aninfrared protocol, a Bluetooth® protocol, a ZigBee protocol, a Z-Waveprotocol, a dedicated short range communications (DSRC) protocol, or awireless universal serial bus (USB) protocol.

Example 36 may include the method of any one of examples 34-35 and/orsome other examples herein, wherein the off-screen input device includesa selected one of a keyboard, a trackpad, a mouse, a track ball, a joystick, or another display screen removably coupled with the displayscreen.

Example 37 may include the method of any one of examples 34-35 and/orsome other examples herein, wherein the on-screen input interfacecontroller is to process the on-screen input in a first time slot or afirst frequency, and to process the off-screen input in a second timeslot or a second frequency.

Example 38 may include the method of any one of examples 34-35 and/orsome other examples herein, wherein the on-screen input device includesa selected one of a user finger, a stylus, or a virtual keyboard.

Example 39 may include one or more computer-readable media havinginstructions for processing inputs to a computing system, upon executionof the instructions by one or more processors, to perform the method ofany one of examples 34-38.

Example 40 may include an apparatus for processing inputs to a computingsystem, comprising: means for receiving an on-screen input provided viaan interaction between an on-screen input device and a display screen ofa computing device; means for processing, by an on-screen inputinterface controller co-located with the display screen, the on-screeninput; means for receiving, by a receiver co-located with the displayscreen, in accordance with a wireless protocol, an off-screen input,wherein the off-screen input is generated by an off-screen input deviceof the computing device, separately located from the display screen; andmeans for processing, by the on-screen input interface controllercoupled to the receiver, the off-screen input.

Example 41 may include the apparatus of example 40 and/or some otherexamples herein, wherein the wireless protocol includes a selected oneof a near field communication (NFC) protocol, a wireless personal areanetwork (WPAN) protocol, a mobile body area networks (MBAN) protocol, aninfrared protocol, a Bluetooth® protocol, a ZigBee protocol, a Z-Waveprotocol, a dedicated short range communications (DSRC) protocol, or awireless universal serial bus (USB) protocol.

Example 42 may include the apparatus of any one of examples 40-41 and/orsome other examples herein, wherein the off-screen input device includesa selected one of a keyboard, a trackpad, a mouse, a track ball, a joystick, or another display screen removably coupled with the displayscreen.

Example 43 may include the apparatus of any one of examples 40-41 and/orsome other examples herein, wherein the on-screen input interfacecontroller is to process the on-screen input in a first time slot or afirst frequency, and to process the off-screen input in a second timeslot or a second frequency.

Example 44 may include the apparatus of any one of examples 40-41 and/orsome other examples herein, wherein the on-screen input device includesa selected one of a user finger, a stylus, or a virtual keyboard.

Although certain embodiments have been illustrated and described hereinfor purposes of description this application is intended to cover anyadaptations or variations of the embodiments discussed herein.Therefore, it is manifestly intended that embodiments described hereinbe limited only by the claims.

What is claimed is:
 1. An apparatus for computing, comprising: anon-screen input interface controller co-located with a display screen ofa computing device, to process an on-screen input provided via aninteraction between an on-screen input device and the display screen;and a receiver co-located with the display screen to wirelessly receivean off-screen input from an off-screen input support device of thecomputing device, separately located from the display screen; whereinthe on-screen input interface controller is to further process theoff-screen input received by the receiver.
 2. The apparatus of claim 1,wherein the receiver is to wirelessly receive the off-screen input inaccordance with a wireless protocol selected from one of a near fieldcommunication (NFC) protocol, a wireless personal area network (WPAN)protocol, a mobile body area networks (MBAN) protocol, an infraredprotocol, a Bluetooth® protocol, a ZigBee protocol, a Z-Wave protocol, adedicated short range communications (DSRC) protocol, or a wirelessuniversal serial bus (USB) protocol.
 3. The apparatus of claim 1,wherein the on-screen input interface controller is to receive theon-screen input in a first time slot or a first frequency, and toreceive the off-screen input in a second time slot or a secondfrequency.
 4. The apparatus of claim 1, wherein the display screen is aselected one of a resistive touchscreen, a capacitive touchscreen, aninfrared touchscreen, an optical touchscreen, a light-emitting diode(LED) display, a liquid crystal display (LCD), a thin film transistorliquid crystal display (TFT-LCD), a digital light processing (DLP)display, a plasma display, an electroluminescent panel, an organiclight-emitting diode (OLED) display, or an electronic paper.
 5. Theapparatus of claim 1, wherein the on-screen input device is a selectedone of a user finger, a stylus, or a virtual keyboard.
 6. The apparatusof claim 1, wherein the off-screen input support device includes: anoff-screen input device to generate the off-screen input, and atransmitter coupled to the off-screen input device to wirelesslytransmit the off-screen input to the receiver.
 7. The apparatus of claim6, wherein the off-screen input device is a selected one of a keyboard,a trackpad, a mouse, a track ball, a joy stick, or another displayscreen removably coupled with the display screen.
 8. The apparatus ofclaim 6, wherein the off-screen input device is a first off-screen inputdevice, and the off-screen input support device further includes: asecond off-screen input device, wherein the second off-screen inputdevice is to generate a second off-screen input to be transmitted by thetransmitter to the receiver through the wireless protocol.
 9. Theapparatus of claim 6, wherein the off-screen input support devicefurther includes an off-screen input interface controller coupled to theoff-screen input device and the transmitter, and the off-screen inputinterface controller is to generate the off-screen input compatible tothe on-screen input.
 10. The apparatus of claim 6, wherein thetransmitter is a touch sensing element located in a location of theoff-screen input support device to control a capacitance of the touchsensing element by a tilting degree of the off-screen input supportdevice with respect to the computing device.
 11. The apparatus of claim6, wherein the transmitter is a first sensing element, the receiver is asecond sensing element capacitively coupled to the first sensingelement, the transmitter is to transmit the off-screen input to thereceiver through capacitive coupling between the first sensing elementand the second sensing element.
 12. The apparatus of claim 6, whereinthe computing device is a computing tablet, the off-screen input supportdevice is a keyboard base, the on-screen input device is a stylus, theoff-screen input device is a keyboard, and the transmitter and thereceiver are two touch sensing elements.
 13. The apparatus of claim 1,wherein the apparatus is the computing device having at least a mainprocessor, the display screen, the on-screen input interface controller,and the receiver, or the computing device having at least a mainprocessor located at the off-screen input support device.
 14. Theapparatus of claim 13, wherein the computing device is a computingtablet.
 15. The apparatus of claim 1, wherein the apparatus is adetachable 2 in 1 device comprising at least the computing device, thedisplay screen, the on-screen input interface controller, and thereceiver.
 16. An apparatus for computing, comprising: an off-screeninput device to generate an off-screen input for a computing device,wherein the off-screen input device is separately located from a displayscreen of the computing device; a transmitter coupled to the off-screeninput device to wirelessly transmit the off-screen input to a receiverco-located with the display screen to receive the off-screen input,wherein the off-screen input is to be processed by an on-screen inputinterface controller co-located with the display screen of the computingdevice.
 17. The apparatus of claim 16, wherein the on-screen inputinterface controller is further to process an on-screen input providedvia an interaction between an on-screen input device and the displayscreen.
 18. The apparatus of claim 16, wherein the off-screen inputdevice is a first off-screen input device, and the apparatus furtherincludes: a second off-screen input device, wherein the secondoff-screen input device is to generate a second off-screen input to betransmitted by the transmitter to the receiver through the wirelessprotocol.
 19. The apparatus of claim 16, further comprising: anoff-screen input interface controller coupled to the off-screen inputdevice and the transmitter, and the off-screen input interfacecontroller is to generate the off-screen input compatible to theon-screen input interface controller.
 20. The apparatus of claim 16,wherein the off-screen input device is a selected one of a keyboard, atrackpad, a mouse, a track ball, a joy stick, or another display screenremovably coupled with the display screen.
 21. The apparatus of claim16, wherein the computing device is a computing tablet, the apparatus isa keyboard base, the off-screen input device is a keyboard, and thetransmitter and the receiver are two touch sensing elements.
 22. Theapparatus of claim 16, wherein the computing device has at least a mainprocessor coupled to the off-screen input device.
 23. The apparatus ofclaim 16, wherein the apparatus is a detachable 2 in 1 device comprisingat least the computing device, the display screen, the on-screen inputinterface controller, the transmitter, and the off-screen input device.24. An apparatus for computing, comprising: an on-screen input interfacecontroller co-located with a display screen of a computing device, and avirtual device interface to operate on the on-screen input interfacecontroller to: process an on-screen input provided via an interactionbetween an on-screen input device and the display screen; and process anoff-screen input provided by an off-screen input support device of thecomputing device, separately located from the display screen.
 25. Theapparatus of claim 24, wherein the virtual device interface is toprocess the on-screen input in a first time slot or a first frequency,and to process the off-screen input in a second time slot or a secondfrequency.